Papers with evaluation fidelity
Can LLMs Narrate Tabular Data? An Evaluation Framework for Natural Language Representations of Text-to-SQL System Outputs (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Text-to-SQL technology bridges natural language (NL) questions and database querying. |
| Approach: | They propose a method for evaluating LLM-generated NLRs using Combo-Eval and a dataset for NLR benchmarking. |
| Outcome: | The proposed method reduces LLM calls by 25-61% and improves performance across scenarios with and without ground truth references. |
GuessArena: Guess Who I Am? A Self-Adaptive Framework for Evaluating LLMs in Domain-Specific Knowledge and Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods for large language models rely on static benchmarks and standardized evaluation protocols. |
| Approach: | They propose an adaptive evaluation framework that integrates dynamic domain knowledge modeling with progressive reasoning assessment to improve evaluation fidelity. |
| Outcome: | Empirical results show that the framework distinguishes LLMs in terms of domain knowledge coverage and reasoning chain completeness. |